{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6f32d781-84ab-4463-b995-ecede1b96e35",
   "metadata": {},
   "source": [
    "# 反向传播推导(迭代)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "02c69702-2d34-4168-8a57-5c3a7ff3a230",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "864cc3ec-991b-4f3b-a627-b134b1664599",
   "metadata": {},
   "source": [
    "## 定义线性方程 \n",
    " - 我的预测的线形方式是 $$y = 2x + 1$$\n",
    " - 为了简化我们只求w，也就是假设b是已知的 $$y' = wx + 1$$\n",
    " - 定义需要预测的线形方程 linreg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a46588d4-384c-4bcd-a092-35639a57ecc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def linreg(x,w):\n",
    "    return w*x + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "896b720c-e5ba-41c5-b657-58c49550bd3d",
   "metadata": {},
   "source": [
    "## 定义损失函数(最小二乘法)\n",
    "\n",
    " $$ L = \\frac{1}{2}(y'-y)^2 $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f4b022eb-ac54-4c58-96ce-3ee977ba9c94",
   "metadata": {},
   "outputs": [],
   "source": [
    "def squared_loss(h_hat,y):\n",
    "    return (h_hat-y)**2/2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba7661da-fa0b-428e-a37e-2e4a744a5a68",
   "metadata": {},
   "source": [
    "## 随机梯度下降"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e5c6e8a5-bee4-42bf-9de1-de2462f74700",
   "metadata": {},
   "outputs": [],
   "source": [
    "def sgd(w, lr): \n",
    "    with torch.no_grad():\n",
    "        w -= lr * w.grad \n",
    "        w.grad.zero_()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed6c799f-9a27-40e5-8b10-8f9827ba9dca",
   "metadata": {},
   "source": [
    "## 用深度学习中术语重新整理赋值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4d1e94ce-8478-474d-b706-5bbc5f7e5ad1",
   "metadata": {},
   "outputs": [],
   "source": [
    "net = linreg\n",
    "loss = squared_loss\n",
    "learn_rate = 0.1 #学习率\n",
    "num_epochs = 5   # 训练轮数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "534e8bb0-c024-44b3-9018-b1d48765dcfe",
   "metadata": {},
   "source": [
    "# 初始化训练数据集\n",
    " - 真实的w是2\n",
    " - 目标函数是 $$ y = 2x + 1 $$\n",
    " - 我们的训练数据集x = [1,2,3,4,5] \n",
    " - 带入目标函数的结果 y = [3,5,7,9,11]\n",
    " - 预测的w初始可以是随机的\n",
    " - torch.rand(1)从区间[0,1)的均匀分布中随机抽取一个随机数生成一个张量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "995a4751-95ed-4482-850a-9406411fc425",
   "metadata": {},
   "outputs": [],
   "source": [
    "true_w = torch.tensor(2)\n",
    "w = torch.rand(1, dtype=torch.float64, requires_grad=True) # requires_grad=True 代表自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "47b25189-1dea-4ecf-8adc-6045707e760d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def synthetic_data(w, num_examples): \n",
    "    X = torch.arange(1,num_examples+1)\n",
    "    y = X*w + 1\n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f93daea7-dacb-4669-b350-0d3437df4392",
   "metadata": {},
   "outputs": [],
   "source": [
    "features, labels = synthetic_data(true_w, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9cf4ffbb-97c7-47da-842f-4985086c89b8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1, 2, 3, 4, 5])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b5090057-9af6-45bf-98f4-3ec5be3bdd00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 3,  5,  7,  9, 11])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9aa64d59-fb69-432c-a3a4-72844c827b2d",
   "metadata": {},
   "source": [
    "## “训练”"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e00115de-e234-455d-a209-9fa846f5bf67",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, l tensor([0.9065], dtype=torch.float64, grad_fn=<DivBackward0>)\n",
      "epoch 1, w tensor([0.7882], dtype=torch.float64, requires_grad=True)\n",
      "epoch 2, l tensor([2.9371], dtype=torch.float64, grad_fn=<DivBackward0>)\n",
      "epoch 2, w tensor([1.2729], dtype=torch.float64, requires_grad=True)\n",
      "epoch 3, l tensor([2.3790], dtype=torch.float64, grad_fn=<DivBackward0>)\n",
      "epoch 3, w tensor([1.9273], dtype=torch.float64, requires_grad=True)\n",
      "epoch 4, l tensor([0.0423], dtype=torch.float64, grad_fn=<DivBackward0>)\n",
      "epoch 4, w tensor([2.0436], dtype=torch.float64, requires_grad=True)\n",
      "epoch 5, l tensor([0.0238], dtype=torch.float64, grad_fn=<DivBackward0>)\n",
      "epoch 5, w tensor([1.9346], dtype=torch.float64, requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "for i in range(num_epochs):\n",
    "    l = loss(net(features[i], w), labels[i]) \n",
    "    print(f\"epoch {i + 1}, l {l}\")\n",
    "    l.backward()\n",
    "    sgd(w, learn_rate) # 使用参数的梯度更新参数\n",
    "    print(f'epoch {i + 1}, w {w}')\n",
    "        \n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d5afc0de-01ba-4324-94e6-e1183b3ffb14",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1.9346], dtype=torch.float64, requires_grad=True)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3928f2f5-92d8-47fd-8d65-9eec09d53998",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5feb1bb-85d8-47cd-8c7d-a03b3f954ef4",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
